Fingerprint-Assisted Force Estimation

ABSTRACT

Embodiments may take the form of a system having a user input device and a first sensor coupled to the user input device. The first sensor is configured to sense touch on a surface of the user input device. The system may also include a second sensor in communication with the surface of the user device configured to sense wetting of a user&#39;s fingerprint on the surface. The system has a processor coupled to the first and second sensors and configured to estimate an amount of force applied by the user&#39;s fingerprint based at least in part upon the sensed wetting of the user&#39;s fingerprint.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/417,164, filed Jan. 25, 2015, and entitled “Fingerprint-AssistedForce Estimation,” which application is a 35 U.S.C. §371 application ofPCT/US2013/032657, which was filed on Mar. 15, 2013, and entitled“Fingerprint-Assisted Force Estimation,” and further claims the benefitunder 35 U.S.C. §119(e) to U.S. provisional application No. 61/676,308,filed Jul. 26, 2012, and entitled, “Fingerprint-Assisted ForceEstimation,” all of which are incorporated by reference as if fullydisclosed herein.

TECHNICAL FIELD

Embodiments are generally related to touch sensitive inputs forelectronic devices and, more particularly, to determining a force of atouch.

BACKGROUND

Touch sensitive inputs are common in today's electronic devices. Inparticular, tablet computers, ebook readers, and smartphones, to name afew, all may rely upon a touch-input display as a primary form of userinput. Other devices, such as trackpads, mice, and so forth may alsoimplement touch sensitive input technology. In such devices, theposition and/or movement of fingers across the surface of the display istranslated as input to the device. Generally, however, the amount ofinformation that may be provided to the device via touch is limited dueto the two-dimensional nature of the touched surface.

SUMMARY

In some embodiments, fingerprint contact with the touch surface is usedto determine the amount of force applied. That is, how well thefingerprint is wetted to the touch surface (e.g., in contact with thetouch surface) is used as one input to determine an applied force.

One embodiment may take the form of a system having a user input deviceand a first sensor coupled to the user input device. The first sensor isconfigured to sense touch on a surface of the user input device. Thefirst sensor may be configured to sense a touch on a surface of the userinput device. The system may also include a second sensor configured tosense a characteristic of a user's fingerprint on the surface. Thesystem may further include a processor coupled to the first and secondsensors and configured to estimate an amount of force applied to thesurface by the user's fingerprint based at least in part upon a ratio ofridges to valleys of the user's fingerprint.

Another embodiment may take the form of a method of approximating force.The method includes storing a calibration value in a non-transitorystorage medium, determining a contact area of a finger on a fingerprintsensor, determining a ratio of ridges to valleys in the contact area,and obtaining force data from a solid state force sensor. Additionally,the method includes determining a force value based upon the force data,the contact area and the ratio.

While multiple embodiments are disclosed, still other embodiments of thepresent invention, will become apparent to those skilled in the art fromthe following Detailed Description. As will be realized, the embodimentsare capable of modifications in various aspects, all without departingfrom the spirit and scope of the embodiments. Accordingly, the drawingsand detailed description are to be regarded as illustrative in natureand not restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example electronic device that may implement afingerprint assisted force determination technique.

FIG. 2 is a block diagram of an example touch I/O device and a hostcomputing system.

FIG. 3. is a block diagram of an example system that includes a touchI/O subsystem.

FIG. 4 is a cross-sectional view of device 1000 taken along line IV-IVin FIG. 1 and illustrating a frustrated total internal reflectionsensor.

FIG. 5A illustrates the same cross-sectional view of FIG. 4 with afinger touching the coverglass.

FIG. 5B is a zoomed in view of the finger touching the coverglass.

FIG. 6 illustrates the same-cross-sectional view of FIG. 4 butillustrates an ultrasonic sensor rather than the frustrated totalinternal reflection sensor.

FIG. 7A illustrates a first layer of the ultrasonic sensor of FIG. 6 ashaving horizontal sensing elements.

FIG. 7B illustrates a second layer of the ultrasonic sensor of FIG. 6 ashaving vertical sensing elements.

FIG. 8 is a cross-sectional view taken along line VIII-VIII in FIG. 1.

FIG. 9 is a plot showing the force sensitivity of a capacitive sensor.

FIG. 10 is a plot comparing the force sensitivity of a capacitive sensorwith that of a frustrated total internal reflection force sensor.

FIG. 11 is a flowchart illustrating an example method for calibratingsolid state force sensor.

DETAILED DESCRIPTION

Embodiments may take the form of systems, methods and apparatuses thatestimate a force applied, by a finger. For example, an, electronicdevice may incorporate a screen, display or cover that senses a user'stouch. The touch screen may capacitively sense the user's touch incertain embodiments. Typically, capacitive-sensing touch screens do notmeasure the force applied to the screen or coverglass but insteadmeasure the presence or absence of a touch, as well as the contact sizeof a touch.

In particular, solid state apparatuses are discussed that measure theinteraction of a finger with a coverglass. The sensing/estimating offorce by a finger allows additional input to be received via a touchinput device. In particular, the force of touch or a change in the forceof touch may be interpreted as an additional input beyond simple touch,multi-touch, contact and/or proximity inputs.

The force measurement may be made through one or more techniques thatmay implement software, hardware and/or firmware beyond that which isimplemented for the touch sensitivity. In some embodiments, fingerprintcontact with the touch surface is used to determine the amount of forceapplied. That is, how well the fingerprint is wetted to the touchsurface is interpreted as an applied force.

One or more technologies may be utilized in the fingerprint-assistedforce sensing. For example, capacitive sensing, frustrated totalinternal reflectance (FTIR), ultrasonic sensing and/or other sensingtechnologies may be utilized. Generally, ultrasonic and/or FTIR sensingtechniques depend upon a transfer of energy out of a device'scoverglass. Generally, more energy transfer out of the coverglassindicates more contact and, therefore, indicates more force beingapplied. The ultrasonic sensing techniques generally sense reflectedenergy. Higher amounts of reflected energy indicate high force. As usedherein, “coverglass” may refer to a cover intended for touch sensingthat is glass, plastic, or another material, clear or opaque.

A fingerprint's wetting (e.g., contact) with the coverglass may varybased upon environmental conditions and biological factors. For example,in humid conditions, ridges and valleys of fingerprints may be moremalleable and may provide better contact whereas dry conditions may leadto the opposite results. Additionally, certain users may have sweaty,moist or wet hands that may provide better wetting while others may havedry hands.

Some embodiments may include a calibration aimed at determining acurrent state of a user's fingerprint. The calibration may be conductedusing a button, such as a home button or other input of an electronicdevice incorporating the touch screen. The button or input may beconfigured with a mechanical actuator that actuates at a known level ofapplied force, The button may further include touch and/or fingerprintsensing elements so that, at the moment the mechanical actuator,actuates, a reading of the touch/fingerprint sensor may be made and theamount of sensed touch correlated with the force of the actuation. Thatis, the amount of sensed touch at the moment of actuation may have aknown force which may serve to calibrate the fingerprint assisted forceestimation. As one example, a mechanical switch (such as a dome switch)located beneath the button may serve as the mechanical actuator. Whenthe button is pressed with sufficient force, the switch collapses. Thetouch may be sensed at the moment of collapse as the force necessary tocollapse the switch remains constant (e.g., the switch always collapsesonce the force exceeds the resistance threshold of the dome switch).

Turning to the drawings and referring initially to FIG. 1, an electronicdevice 1000 is illustrated which may implement fingerprint assistedforce measurements. The illustrated electronic device 1000 is a smartphone, such as the iPhone® from Apple, Inc. However, it should beappreciated that various other types of devices may implement thefingerprint assisted force measurement techniques described herein. Forexample, a notebook computer, table computers, desktop computers,trackpads and so forth, all may implement fingerprint assisted forcemeasurement techniques. The electronic device 1000 includes a display1002 which may take the form of a touch screen device, such as acapacitive touch screen, or any other suitable display device.

Additionally, the device 1000 includes one or more buttons 1004 and/orother input devices. In some embodiments, the button 1004 may take theform of a home button and may be utilized as a calibration tool for thefingerprint assisted force measurements. As described in greater detailbelow, the button 1004 may include an element having a known forcefeature that may be utilized in conjunction with the fingerprint sensingto self-calibrate the device 1000.

Generally, the display 1002 and/or the button 1004 may includetouch-sensitive input/output (I/O) devices. As one example, afingerprint array sensor may be positioned beneath the button. Thefingerprint sensor may capture an image of the finger when the button istouched and/or the mechanical switch located beneath the buttoncollapses. It should be appreciated that the operation of a fingerprintsensor is generally known in the art.

FIG. 2 illustrates an example block diagram showing an exampleembodiment including touch I/O device 1006 that can receive touch inputfor interacting with a computing system 1008. The communication may bevia a wired or wireless communication channel 1010. Touch I/O device1006 may be used to provide user input to computing system 1008 in lieuof or in combination with other input devices such as a keyboard, mouse,etc. One or more touch I/O devices 1006 may be used for providing userinput to computing system 1008. Touch I/O device 1006 may be an integralpart of computing system 1008 (e.g., touch screen on a laptop) or may beseparate from computing system 1008.

Touch I/O device 1006 may include a touch sensitive panel which iswholly or partially transparent, semitransparent, non-transparent,opaque or any combination thereof. Touch I/O device 1006 may be embodiedas a touch screen, touch pad, a touch screen functioning as a touch pad(e.g., a touch screen replacing the touchpad of a laptop), a touchscreen or touchpad combined or incorporated with any other input device(e.g., a touch screen or touchpad disposed on a keyboard) or anymulti-dimensional object having a touch sensitive surface for receivingtouch input.

In one example, touch I/O device 1006 embodied as a touch screen mayinclude a transparent and/or semitransparent touch sensitive panelpartially or wholly positioned over at least a portion of a display.According to this embodiment, touch I/O device 1006 functions to displaygraphical data transmitted from computing system 1008 (and/or anothersource) and also functions to receive user input. In other embodiments,touch I/O device 1006 may be embodied as an integrated touch screenwhere touch sensitive components/devices are integral with displaycomponents/devices. In still other embodiments a touch screen may beused as a supplemental or additional display screen for displayingsupplemental or the same graphical data as a primary display and toreceive touch input.

Touch I/O device 1006 may be configured to detect the location of one ormore touches or near touches on device 1006 based on capacitive,resistive, optical, acoustic, inductive, mechanical, chemicalmeasurements, or any phenomena that can be measured with respect to theoccurrences of the one or more touches or near touches in proximity todevice 1006. Software, hardware, firmware or any combination thereof maybe used to process the measurements of the detected touches to identifyand track one or more gestures. A gesture may correspond to stationaryor non-stationary, single or multiple, touches or near touches on touchI/O device 1006. A gesture may be performed by moving one or morefingers or other objects in a particular manner on touch I/O device 1006such as tapping, pressing, rocking, scrubbing, twisting, changingorientation, pressing with varying pressure and the like at essentiallythe same time, contiguously, or consecutively. A gesture may becharacterized by, but is not limited to a pinching, sliding, swiping,rotating, flexing, dragging, or tapping motion between or with any otherfinger or fingers. A single gesture may be performed with one or morehands, by one or more users, or any combination thereof.

Computing system 1008 may drive a display with graphical data to displaya graphical user interface (GUI). The GUI may be configured to receivetouch input via touch I/O device 1006. Embodied as a touch screen, touchI/O device 1006 may display the GUI. Alternatively, the GUI may bedisplayed on a display separate from touch I/O device 1006.

The GUI may include graphical elements displayed at particular locationswithin the interface. Graphical elements may, include but are not,limited to, a variety of displayed virtual input devices includingvirtual scroll wheels, a virtual keyboard, virtual knobs, virtualbuttons, any virtual UI, and the like. A user may perform gestures atone or more particular locations on touch I/O device 1006 which may beassociated with the graphical elements of the GUI. In other embodiments,the user may perform gestures at one or more locations that areindependent of the locations of graphical elements of the GUI. Gesturesperformed on touch I/O device 1006 may directly or indirectlymanipulate, control, modify, move, actuate, initiate or generally affectgraphical elements such as cursors, icons, media files, lists, text, allor portions of images, or the like within the GUI. For instance, in thecase of a touch screen, a user may directly interact with a graphicalelement by performing a gesture over the graphical element on the touchscreen. Alternatively, a touch pad generally provides indirectinteraction. Gestures may also affect non-displayed GUI elements (e.g.,causing user interfaces to appear) or may affect other actions withincomputing system 1008 (e.g., affect a state or mode of a GUI,application, or operating system). Gestures may or may not be performedon touch I/O device 1006 in conjunction with a displayed cursor. Forinstance, in the case in which gestures are performed on a touchpad, acursor (or pointer) may be displayed on a display screen or touch screenand the cursor may be controlled via touch input on the touchpad tointeract with graphical objects on the display screen. In otherembodiments in which gestures are performed directly on a touch screen,a user may interact directly with objects on the touch screen, with orwithout a cursor or pointer being displayed on the touch screen.

Feedback may be provided to the user via communication channel 1010 inresponse to or based on the touch or near touches on touch I/O device1006. Feedback may be transmitted optically, mechanically, electrically,olfactory, acoustically, or the like or any combination thereof and in avariable or non-variable manner.

Attention is now directed towards embodiments of a system architecturethat may be embodied within any portable or non-portable deviceincluding but not limited to a communication device (e.g. mobile phone,smart phone), a multi-media device (e.g., MP3 player, TV, radio), aportable or handheld computer (e.g., tablet, netbook, laptop), a desktopcomputer, an All-In-One desktop, a peripheral device, or any othersystem or device adaptable to the inclusion of system architecture 2000,including combinations of two or more of these types of devices. FIG. 3is a block diagram of one embodiment of system 2000 that generallyincludes one or more computer-readable mediums 2001, processing system2004, Input/Output (I/O) subsystem 2006, radio frequency (RF) circuitry2008 and audio circuitry 2010. These components may be coupled by one ormore communication buses or signal lines 2003.

It should be apparent that the architecture shown in FIG. 3 is only oneexample architecture of system 2000, and that, system 2000 could havemore or fewer components than shown, or a different configuration ofcomponents. The various components shown in FIG. 3 can be implemented inhardware, software, firmware or any combination thereof, including oneor more signal processing and/or application specific integratedcircuits.

RF circuitry 2008 is used to send and receive information over awireless link or network to one or more other devices and includeswell-known circuitry for performing this function. RF circuitry 2008 andaudio circuitry 2010 are coupled to processing system 2004 viaperipherals interface 2016. Interface 2016 includes various knowncomponents for establishing and maintaining communication betweenperipherals and processing system 2004. Audio circuitry 2010 is coupledto audio speaker 2050 and microphone 2052 and includes known circuitryfor processing voice signals received from interface 2016 to enable auser to communicate in real-time with other users. In some embodiments,audio circuitry 2010 includes a headphone jack (not shown).

Peripherals interface 2016 couples the input and output peripherals ofthe system to processor 2018 and computer-readable medium 2001. One ormore processors 2018 communicate with one or more computer-readablemediums 2001 via controller 2020. Computer-readable medium 2001 can beany device or medium that can store code and/or data for use by one ormore processors 2018. Medium 2001 can include a memory hierarchy,including but not limited to cache, main memory and secondary memory.The memory hierarchy can be implemented using any combination of RAM(e.g., SRAM, DRAM, DDRAM), ROM, FLASH, magnetic and/or optical storagedevices, such as disk drives, magnetic tape, CDs (compact disks) andDVDs (digital video discs). Medium 2001 may also include a transmissionmedium for carrying information-bearing signals indicative of computerinstructions or data (with or without a carrier wave upon which thesignals are modulated). For example, the transmission medium may includea communications network, including but not limited to the Internet(also referred to as the World Wide Web), intranet(s), Local AreaNetworks (LANs), Wide Local Area Networks (WLANs), Storage Area Networks(SANs), Metropolitan Area Networks (MAN) and the like.

One or more processors 2018 run various software components stored inmedium 2001 to perform various functions for system 2000. In someembodiments, the software components include operating system 2022,communication module (or set of instructions) 2024, touch processingmodule (or set of instructions) 2026, graphics module (or set ofinstructions) 2028, one or more applications (or set of instructions)2030, and fingerprint force module (or set of instructions) 2038. Eachof these modules and above noted applications correspond to a set ofinstructions for performing one or more functions described above andthe methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (i.e., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwisere-arranged in various embodiments. In some embodiments, medium 2001 maystore a subset of the modules and data structures identified above.Furthermore, medium 2001 may store additional modules and datastructures not described above.

Operating system 2022 includes various procedures, sets of instructions,software components and/or drivers for controlling and managing generalsystem tasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 2024 facilitates communication with other devicesover one or more external ports 2036 or via RF circuitry 2008 andincludes various software components for handling data received from RFcircuitry 2008 and/or external port 2036.

Graphics module 2028 includes various known software components forrendering, animating and displaying graphical objects on a displaysurface. In embodiments in which touch I/O device 2012 is a touchsensitive display (e.g., touch screen), graphics module 2028 includescomponents for rendering, displaying, and animating objects on the touchsensitive display.

One or more applications 2030 can include any applications installed onsystem 2000, including without limitation, a browser, address book,contact list, email, instant messaging, word processing, keyboardemulation, widgets, JAVA-enabled applications, encryption, digitalrights management, voice recognition, voice, replication, locationdetermination capability (such as that provided by the globalpositioning system (GPS)), a music player, etc.

Touch processing module 2026 includes various software components forperforming various tasks associated with touch I/O device 2012 includingbut not limited to receiving and processing touch input received fromI/O device 2012 via touch I/O device controller 2032.

System 2000 may further include fingerprint force module 2038 for,performing the method/functions as described herein in connection withFIGS. 4-11. Fingerprint force module 2038 may at least function todetermine if a force threshold has been exceeded. Module 2038 may alsointeract with applications, software, hardware and/or other deviceswithin the system 2000. Module 2038 may be embodied as hardware,software, firmware, or any combination thereof. Although module 2038 isshown to reside within medium 2001, all or portions of module 2038 maybe embodied within other components within system 2000 or may be whollyembodied as a separate component within system 2000.

I/O subsystem 2006 is coupled to touch I/O device 2012 and one or moreother I/O devices 2014 for controlling or performing various functions.Touch I/O device 2012 communicates with processing system 2004 via touchI/O device controller 2032, which includes various components forprocessing user touch input (e.g., scanning hardware). A fingerprintsensor 2042 and a fingerprint controller 2044 may also be included toreceive and communicate the fingerprint sensing with the processingsystem. One or more other input controllers 2034 receives/sendselectrical signals from/to other I/O devices 2014. Other I/O devices2014 may include physical buttons, dials, slider switches, sticks,keyboards, touch pads, additional display screens, or any combinationthereof.

If embodied as a touch screen, touch I/O device 2012 displays visualoutput to the user in a GUI. The visual output may include text,graphics, video, and any combination thereof. Some or all of the visualoutput may correspond to user-interface objects. Touch I/O device 2012forms a touch-sensitive surface that accepts touch input from the user.Touch I/O device 2012 and touch screen controller 2032 (along with anyassociated modules and/or sets of instructions in medium 2001) detectsand tracks touches or near touches (and any movement or release of thetouch) on touch I/O device 2012 and converts the detected touch inputinto interaction with graphical objects, such as one or moreuser-interface objects. In the case in which device 2012 is embodied asa touch screen, the user can directly interact with graphical objectsthat are displayed on the touch screen. Alternatively, in the case inwhich device 2012 is embodied as a touch device other than a touchscreen (e.g., a touch pad), the user may indirectly interact withgraphical objects that are displayed on a separate display screenembodied as I/O device 2014.

Touch I/O device 2012 may be analogous to the multi-touch sensitivesurface described in the following U.S. Pat. No. 6,323,846 (Westerman etal.), U.S. Pat. No. 6,570,557 (Westerman et al), and/or U.S. Pat. No.6,677,932 (Westerman), and/or U.S. Patent Publication 2002/0015024A1,each of which is hereby incorporated by reference.

Embodiments in which touch I/O device 2012 is a touch screen, the touchscreen may use LCD (liquid crystal display) technology, LPD (lightemitting polymer display) technology, OLED (organic LED), or OEL(organic electro luminescence), although other display technologies maybe used in other embodiments.

Feedback may be provided by touch I/O device 2012 based on the user'stouch input as well as a state or states of what is being displayedand/or of the computing system. Feedback may be transmitted optically(e.g., light signal or displayed image), mechanically (e.g., hapticfeedback, touch feedback, force feedback, or the like), electrically(e.g., electrical stimulation), olfactory, acoustically (e.g., beep orthe like), or the like or any combination thereof and, in a_(—) variableor non-variable manner.

The I/O subsystem 2006 may include and/or be coupled to one or moresensors configured to be utilized in the force determination. Inparticular, the I/O subsystem 2006 may include an LED 3002 and a sensor3004, and/or an ultrasonic sensor 4000. Each of the LED 3002, sensor3004 and ultrasonic sensor 4000 may be coupled to the touch I/O devicecontroller 2032, or another I/O controller (not shown). The LED 3002,sensor 3004 and ultrasonic sensor 4000 will each be discussed in greaterdetail below.

System 2000 also includes power system 2044 for powering the varioushardware components and may include a power management system, one ormore power sources, a recharging system, a power failure detectioncircuit, a power converter or inverter, a power status indicator and anyother components typically associated with the generation, managementand distribution of power in portable devices.

In some embodiments, peripherals interface 2016, one or more processors2018, and memory controller 2020 may be implemented on a single chip,such as processing system 2004. In some other embodiments, they may beimplemented on separate chips.

Turning to FIG. 4, a cross-sectional view taken along line in FIG. 1illustrates the FTIR solid state sensor. In the FTIR technique,electromagnetic energy 3001, such as light, is directed into thecoverglass 3000. The electromagnetic energy 3001 may be provided by anysuitable device. For example, a light emitting diode 3002 may be used toprovide the electromagnetic energy. A conduit such as a light guide mayhelp inject the light into coverglass 3000. A sensor 3004 is provided tosense the level of electromagnetic energy in the coverglass 3000. Aconduit such as a light guide 3006 may help extract the electromagneticenergy from the coverglass 3000. A touch sensor 3008, such as acapacitive touch sensor described above, may be located below thecoverglass to sense touch.

The FTIR technique senses attenuation of the electromagnetic energy 3001in the coverglass. That is, the coverglass 3000 may generally have anindex of refraction different from that of air which may contact thecoverglass. Specifically, the air or other substance in contact with thecoverglass has a lower index of refraction than that of the coverglass3000. For example, the coverglass may have an index of refraction at,near or above 1.4, whereas air may have an index of refraction ofapproximately 1.0. The electromagnetic energy 3000 is inserted into thecoverglass so that it has an angle of incidence greater than the“critical angle.” The critical angle is generally an angle relative to anormal to a boundary interface below which refraction occurs and abovewhich total reflection occurs.

Due to the difference in the index of refraction and the angle ofincidence, the electromagnetic energy inserted, into the coverglassexperiences total internal reflection.

That is, none of the electromagnetic energy refracts through theboundary of the coverglass. When a finger touches the coverglass,however, the electromagnetic energy refracts, thereby attenuating theelectromagnetic energy reflected within the coverglass. The attenuationmay be determined and correlated with an amount of force being applied.

Generally, a lighter touch will result in a lower attenuation. This isdue in part to the total surface area of the finger that makes contactwith the coverglass surface and allows the electromagnetic energy torefract. With a light touch, only the top of a fingerprint's ridges makecontact with the coverglass surface. Contrastingly, when more force isapplied, a greater surface area of the ridges will contact thecoverglass surface and, thereby, result in greater attenuation throughrefraction.

FIG. 5A illustrates a finger touching 3010 the surface 3012 of thecoverglass 3000. FIG. 5B is a zoomed-in view of the finger touching thesurface. As shown, the ridges of the fingerprint make contact with thesurface of the coverglass and allow electromagnetic energy to escapethrough the boundary of the coverglass. The valleys of the fingerprint,however, do not make contact and light does not escape via the valleys.An attenuated signal 3001′ remains within the coverglass 3000. Theattenuated signal 3001′ is sensed and a determination of force may bemade based on the attenuated signal.

Turning to FIG. 6, a cross-section view taken along line IV-IV of FIG. 1is illustrated as having an ultrasonic sensor 4000 coupled thereto tosense force, in accordance with an alternative embodiment. Theultrasonic sensor 4000 may be located under the capacitive touch sensor3008. The ultrasonic sensor 4000 transmits a signal up through thecoverglass 3000 and then senses reflected signals. When a finger istouching the coverglass, the finger will reflect signal. For theultrasonic sensor embodiment, a display without an air gap may beimplemented. For example, an organic light emitting diode (OLED) displayor liquid crystal display (LCD) may be implemented.

The ultrasonic sensor 4000 may take any suitable form and in someembodiments may take the form of a piezoelectric sensor that is actuatedby applying a voltage. The ultrasonic sensor 4000 may include multiplelayers. FIGS. 7A and 7B illustrate two layers of the sensor.Specifically, FIG. 7A illustrates a first layer 4002 having horizontallyarranged sensors 4004. FIG. 7B illustrates a second layer 4006 havingvertically arranged sensors 4008, or sensors arranged cross-wise fromthose of the first layer 4002. The first and second layers 4002, 4006are stacked to form the sensor. Generally, in one embodiment forexample, a pattern of pulses may be propagated through columns andreflected on the rows.

That is, the columns may be ultrasonic drivers while the rows areultrasonic sensors, or vice-versa. In some embodiments, there may, bemore ultrasonic sensors than drivers

In some embodiments, the ultrasonic sensor 4000 may be configured witheach sensor spaced five millimeters apart in both the vertical andhorizontal direction so that the sensor has five millimeter by fivemillimeter pixels. In some embodiments, the ultrasonic sensor may notable to recognize a fingerprint, but could estimate where energy isbeing absorbed so that force may be approximated. That is, the spacingbetween sensors may provide a gross measurement of contact between thefinger and the coverglass; this measurement may lack resolution todetect individual ridges and valleys of a fingerprint but may detect thesize of the contact area with the coverglass (within the resolution ofthe sensors). The size of that contact patch may serve as a proxy forthe force exerted by the user, as larger contact patches may equate to alarger exerted force, all other things being equal. A larger the contactarea generally indicates a higher force. The system can have severaldifferent size-to-force curves stored in memory, possibly as data pointsin a look-up table. The curves may represent fingers having differentwetting characteristics. The contact patch created when the dome switchbeneath the home button collapses can be measured through thefingerprint sensor by evaluation of a ratio of the ridges to thevalleys, as the size and ratio determine the amount of flesh that makescontact within the contact patch. The higher this ratio is the wetterthe user's skin is. That ratio can be used to figure out which of thesize-to-force curves to use for later touches on the coverglass. Thus,compensation may be made for the wetting of the person's skin and moreaccurately measure force, since a single reading that gives wettinginformation at a known force may be used as a calibration for subsequentforce readings.

For each of the FTIR and ultrasonic techniques, fingerprints may varybased on temperature, moisture content, environmental conditions and soon, and this will impact how well the fingerprint is wetted to thecoverglass. That is, environmental and/or biological factors may affecthow well the fingerprint couples to the surface of the coverglass. Assuch, a calibration may be performed for each session of use of thefingerprint assisted force determination that includes an estimation ofthe condition of the finger.

FIG. 8 is a cross-sectional view taken along line VIII-VIII in FIG. 1.The cross-section shows the structure of the button 1004. Specifically,the button 1004 includes a touch sensing element 5002 and a forcethreshold element 5004. The touch sensing element 5002 may take anysuitable form and in some embodiments may include a capacitive touchsensor array. In other embodiments, the touch sensing element may takethe form of an FTIR touch sensor, an ultrasonic sensor, or anycombination of such sensors.

The force threshold element 5004 may be configured to actuate uponreceiving a threshold amount of force. For example, the force thresholdelement may take the form of a dome or buckling switch that buckles uponapplication of a threshold force. The threshold force for actuation ofthe force threshold element 5004 may be set at any reasonable level. Insome embodiments, the threshold force may be set to approximately 200grams. That is the buckling switch may buckle when approximately 200grams of force are applied. In other embodiments, the force thresholdmay be greater than or less than 200 grams force.

The touch sensing element 5002 may operate concurrently and inconjunction with the force threshold element 5004 as a calibrating tool.Specifically, at the moment that the force threshold element 5004actuates, the touch sensing element 5002 may obtain a reading indicatingthe amount of contact of the finger at the threshold force level. Thisinformation may be used to know how well the finger wets with a surfaceand, in particular, the amount of force being applied when similarreadings are achieved.

As the button 1004 may have a curved surface with which a user's fingerinterfaces, the touch measurements may be slightly skewed. As such,touch sensing data may be appropriately modified to translate thereadings for use on a flat surface. In some embodiments, only the centerportion of the button is used to read in the touch data. In otherembodiments, the edge portion of the button is used to read in the touchdata. In still other embodiments, a filter may be applied to limit theeffects of the curved surface on the touch readings.

The data read by the touch sensor 5002 may be stored and utilized tomake determinations of force applied to the touch screen. In particular,the touch sensor data may be used as an indictor of fingerprint wetting.That is, how well a person's fingerprint wets to the touch screen. Asindividual biological and environmental factors may influence how well aparticular user's finger wets to the touch screen, the calibrationprovided by the touch sensor data may help to achieve more accurateforce sensing results. For example, when using an FTIR sensor, inparticularly humid conditions the user's finger may wet well with thesurface and thus the amount of attenuation will be greater than underdry conditions. The calibration step will help to recognize that ahigher amount of attenuation may be expected and avoid false positivesfor exceeding an actuation force level, Alternatively, reflected signalsdetected by the ultrasonic sensor may increase under humid conditions.

Generally, where a capacitive sensor is used as a calibration sensor,there may be two parameters of interest. Specifically, the twoparameters of interest may include: (1) the average capacitance betweena capacitive array and a finger, and (2) the variation of capacitancebetween fingerprint ridge and finger print valley. The first parametermay be used as a baseline in determining if a particular fingerprintwets better or more poorly than average. The second parameter is relatedto the first parameter but allows a determination as to whether theridges alone or the ridges and valleys are in contact with, the targetsurface. With these two parameters it may be determined that if thesignal capacitance is high then the person has good flexible prominentfingerprint that wets well to the surface (e.g., there will be a highsignal voltage at moment of 200 grams of force). Hence, in the FTIRexample, there will be a high attenuation of signal when user touchescoverglass and the signal should be adjusted or scaled accordingly.

FIG. 9 is a plot 6000 illustrating force sensitivity for a capacitivesensor under wet and dry conditions. For example, the plot may representreadings obtained from a user pressing on a home button. Thus, the plot6000 may be used to help calibrate force estimations of subsequenttouches on a cover glass. The solid line 6002 represents the wetconditions and the dotted line 6004 represents dry conditions.Additionally, the vertical axis 6006 represents a change in signalstrength and the horizontal axis 6008 represents the amount of forceapplied. As shown, the capacitive sensor has a significant change insignal prior to a finger making contact with the surface. The signalflattens as force is applied in both wet and dry conditions.

Additionally, a mechanical actuator may actuate upon receiving a knownforce level. For example, the mechanical actuator may actuate atapproximately 200 grams. Using the curves, the signal characteristicswhen the mechanical actuator actuates may be used to know if a user'stouch registers on the “wet” curve 6002 or the “dry” curve 6004, orsomewhere in between. With that information, the appropriate curve inFIG. 10 may be selected to generate a more accurate proxy for force. Itshould be appreciated that the wet and dry curves are related to afinger's wetting characteristics. The environment may influence thischaracteristic, but it's only one factor of several or many.

FIG. 10 is a plot 7000 illustrating force sensitivity for the capacitivesensor and other sensors for the purpose of comparison. The capacitivesensor plot 6002 is the same as illustrated in FIG. 9. Both wet (7002)and dry (7004) plots of the FTIR sensor are shown. In contrast to thecapacitive sensor, the FTIR sensor shows sharp increases in signalchanges after contact with the surface. Under dry conditions, the plotline 7004 has a shallower slope than under wet conditions. However, evenunder dry conditions, the FTIR sensor demonstrates significant signalchanges induced by force changes. As such, the FTIR and ultrasonicsensors generally have better sensitivity to force than a capacitivesensor. Ultrasonic sensors will generally demonstrate plotcharacteristics similar to those of the FTIR sensor. It should be noted,however, that FIG. 10 shows merely generalized representations offorce-to-signal curves. The curves may vary depending on whether FTIR orultrasonics are, used, the configuration of the emitters and detectors,and so forth.

Generally, an object of using the plot 6000 of FIG. 9 with the curves7002, 7004 is to use the data captured by the fingerprint sensor whenthe button collapses to help improve a force estimation of touches on acover glass. If the data shows that the person has a “wet” fingerprint,and so is on curve 6002 of FIG. 9 when the button collapses, then curve7002 may be used when the user touches the coverglass to approximateforce. If the user was on curve 6004 when the button collapses, thencurve 7004 is used for coverglass touches. As such, the readings fromone sensor that has a known force activation point, and, is associatedwith a first input device (e.g., fingerprint scanner), is used tocalibrate a completely different sensor associated with a completelydifferent input device (e.g., touch screen).

FIG. 11 is a flowchart illustrating an example method 8000 forimplementing the fingerprint assisted force determination. The methodbegins by sensing a touch on the home button (Block 8002). Upon reachinga threshold level of force, the force threshold element actuates (Block8004) and a touch sensor is read (Block 8006). The reading is then usedas a calibration for force sensing on the coverglass (Block 8008). Insome embodiments, a look up table may be provided that correlates thesensor reading with those that may be read from the coverglass.

The use of a lookup table may allow for estimation of force based onreadings from the sensors associated with the coverglass. Thisinformation may then be used to determine when a user changes the amountof force that is being applied, for example, when making a selection,Furthermore, in some embodiments, as the amount of force increases,there may be one or more actuation thresholds that may be applied. Eachforce threshold may have a different action associate with it. Forexample, upon reaching a first force threshold (e.g. 200 grams) an area,icon or word may be highlighted or selected. Upon reaching a secondforce threshold (e.g. 400 grams) an action may be taken, such asautocorrect of a misspelled word, routing to a linked website, orexecuting a command related to an icon. As such, a diversity offunctionality may be provided through multiple levels of forcethresholds.

The foregoing describes some example techniques using fingerprints inforce estimation. Although the foregoing discussion has presentedspecific embodiments, persons skilled in the art will recognize thatchanges may be made in form and detail without departing from the spiritand scope of the embodiments. For example, it should be appreciated thatinformation obtained through the calibration steps may be used to scalemeasured force data to obtain a force value For example, the calibrationmay simply be a multiplier obtained from an initial wettingdetermination step. Furthermore, in some embodiments, a total area of afingerprint may be used alone or in conjunction with one or more othermeasured or estimated parameters. For example, a capacitive touch sensormay be used to determine the total area of a fingerprint and/or avariance in the size of the area and force may be estimated based on thearea or change in area. Accordingly, the specific embodiments describedherein should be understood as examples and not limiting the scopethereof.

1. A system comprising: a user input device; a first sensor coupled tothe user input device, wherein the first sensor is configured to sensetouch on a surface of the user input device; a second sensor configuredto sense a characteristic of a user's fingerprint on the surface; aprocessor coupled to the first and second sensors and configured toestimate an amount of force applied to the surface by the user's fingerbased at least in part upon the sensed characteristic of the user'sfingerprint, said characteristic measured by an evaluation of ridges tovalleys within a contact area in the user's fingerprint.
 2. The systemof claim 1, wherein the second sensor comprises one of a capacitivetouch sensor, a frustrated total internal reflection sensor, or anultrasonic sensor.
 3. The system of claim 1, wherein the first sensorcomprises a capacitive touch sensor.
 4. The system of claim 1, whereinthe user input device comprises a touch screen display device.
 5. Thesystem of claim 4, wherein the touch screen display device comprises oneof an organic light emitting diode device or a liquid crystal displaydevice.
 6. The system of claim 1, wherein the user input devicecomprises one of a trackpad, a mouse, or a touch screen.
 7. The systemof claim 1, further comprising a force calibration system.
 8. The systemof claim 7, wherein the force calibration system comprises: a mechanicalactuator; and a touch sensor.
 9. The system of claim 8, wherein themechanical actuator comprises a button having a known force threshold.10. The system of claim 9, wherein the known force threshold isapproximately 200 grams.
 11. The system of claim 8, wherein the touchsensor comprises one of a capacitive touch sensor, a frustrated totalinternal reflectance sensor or an ultrasonic sensor.
 12. The system ofclaim 7, wherein the force calibration system is integrated into a homebutton of the device.
 13. The system of claim 7 further comprising alook up table, the look up table correlating a calibration valueobtained from the calibration system with values sensed by the secondsensor to estimate an amount of force applied to the user input device.14. A method of approximating force comprising: storing a calibrationvalue in a non-transitory storage medium; determining a contact area ofa finger on a fingerprint sensor; determining a ratio of ridges tovalleys in the contact area; obtaining force data from a solid stateforce sensor; and determining a force value based upon the force data,the contact area and the ratio.
 15. The method of claim 14 furthercomprising: operating a mechanical actuator, the mechanical actuatorhaving a known force threshold; and capturing the calibration value byreading data from a touch sensor associated with the mechanicalactuator.
 16. The method of claim 14, wherein obtaining force datacomprises operating at least one of: a capacitive touch sensor; afrustrated total internal reflectance sensor; or an ultrasonic sensor.17. The method of claim 14, wherein determining a force value comprisesreferencing a look-up table.
 18. The method of claim 14, whereindetermining a force value comprises computing a force value by scalingthe force data based on the calibration value.
 19. The method of claim14 further comprising replacing the stored calibration data uponinitiation of each unique session.
 20. The method of claim 15, whereinthe known force threshold is approximately 200 grams.